Machine Translation between Spoken Languages and Signed Languages Represented in SignWriting

This paper presents work on novel machine translation (MT) systems between spoken and signed languages, where signed languages are represented in SignWriting, a sign language writing system. Our work seeks to address the lack of out-of-the-box support for signed languages in current MT systems and is based on the SignBank dataset, which contains pairs of spoken language text and SignWriting content. We introduce novel methods to parse, factorize, decode, and evaluate SignWriting, leveraging ideas from neural factored MT. In a bilingual setup—-translating from American Sign Language to (American) English—-our method achieves over 30 BLEU, while in two multilingual setups—-translating in both directions between spoken languages and signed languages—-we achieve over 20 BLEU. We find that common MT techniques used to improve spoken language translation similarly affect the performance of sign language translation. These findings validate our use of an intermediate text representation for signed languages to include them in natural language processing research

Linguistically Motivated Sign Language Segmentation

Sign language segmentation is a crucial task in sign language processing systems. It enables downstream tasks such as sign recognition, transcription, and machine translation. In this work, we consider two kinds of segmentation: segmentation into individual signs and segmentation into phrases, larger units comprising several signs. We propose a novel approach to jointly model these two tasks. Our method is motivated by linguistic cues observed in sign language corpora. We replace the predominant IO tagging scheme with BIO tagging to account for continuous signing. Given that prosody plays a significant role in phrase boundaries, we explore the use of optical flow features. We also provide an extensive analysis of hand shapes and 3D hand normalization. We find that introducing BIO tagging is necessary to model sign boundaries. Explicitly encoding prosody by optical flow improves segmentation in shallow models, but its contribution is negligible in deeper models. Careful tuning of the decoding algorithm atop the models further improves the segmentation quality. We demonstrate that our final models generalize to out-of-domain video content in a different signed language, even under a zero-shot setting. We observe that including optical flow and 3D hand normalization enhances the robustness of the model in this context.Comment: Accepted at EMNLP 2023 (Findings

Considerations for meaningful sign language machine translation based on glosses

Automatic sign language processing is gaining popularity in Natural Language Processing (NLP) research (Yin et al., 2021). In machine translation (MT) in particular, sign language translation based on glosses is a prominent approach. In this paper, we review recent works on neural gloss translation. We find that limitations of glosses in general and limitations of specific datasets are not discussed in a transparent manner and that there is no common standard for evaluation. To address these issues, we put forward concrete recommendations for future research on gloss translation. Our suggestions advocate awareness of the inherent limitations of gloss-based approaches, realistic datasets, stronger baselines and convincing evaluation

Considerations for meaningful sign language machine translation based on glosses

Automatic sign language processing is gaining popularity in Natural Language Processing (NLP) research (Yin et al., 2021). In machine translation (MT) in particular, sign language translation based on glosses is a prominent approach. In this paper, we review recent works on neural gloss translation. We find that limitations of glosses in general and limitations of specific datasets are not discussed in a transparent manner and that there is no common standard for evaluation. To address these issues, we put forward concrete recommendations for future research on gloss translation. Our suggestions advocate awareness of the inherent limitations of gloss-based approaches, realistic datasets, stronger baselines and convincing evaluation

Automatic Sound Event Detection and Classification of Great Ape Calls Using Neural Networks

We present a novel approach to automatically detect and classify great ape calls from continuous raw audio recordings collected during field research. Our method leverages deep pretrained and sequential neural networks, including wav2vec 2.0 and LSTM, and is validated on three data sets from three different great ape lineages (orangutans, chimpanzees, and bonobos). The recordings were collected by different researchers and include different annotation schemes, which our pipeline preprocesses and trains in a uniform fashion. Our results for call detection and classification attain high accuracy. Our method is aimed to be generalizable to other animal species, and more generally, sound event detection tasks. To foster future research, we make our pipeline and methods publicly available.Comment: Accepted at ICPhS 2023 (Poster

An Open-Source Gloss-Based Baseline for Spoken to Signed Language Translation

Sign language translation systems are complex and require many components. As a result, it is very hard to compare methods across publications. We present an open-source implementation of a text-to-gloss-to-pose-to-video pipeline approach, demonstrating conversion from German to Swiss German Sign Language, French to French Sign Language of Switzerland, and Italian to Italian Sign Language of Switzerland. We propose three different components for the text-to-gloss translation: a lemmatizer, a rule-based word reordering and dropping component, and a neural machine translation system. Gloss-to-pose conversion occurs using data from a lexicon for three different signed languages, with skeletal poses extracted from videos. To generate a sentence, the text-to-gloss system is first run, and the pose representations of the resulting signs are stitched together

Polydopamine-Decorated Microcomposites Promote Functional Recovery of an Injured Spinal Cord by Inhibiting Neuroinflammation

Neuroinflammation following spinal cord injury usually aggravates spinal cord damage. Many inflammatory cytokines are key players in neuroinflammation. Owing largely to the multiplicity of cytokine targets and the complexity of cytokine interactions, it is insufficient to suppress spinal cord damage progression by regulating only one or a few cytokines. Herein, we propose a two-pronged strategy to simultaneously capture the released cytokines and inhibit the synthesis of new ones in a broad-spectrum manner. To achieve this strategy, we designed a core/shell-structured microcomposite, which was composed of a methylprednisolone-incorporated polymer inner core and a biocompatible polydopamine outer shell. Thanks to the inherent adhesive nature of polydopamine, the obtained microcomposite (MP-PLGA@PDA) efficiently neutralized the excessive cytokines in a broad-spectrum manner within 1 day after spinal cord injury. Meanwhile, the controlled release of immunosuppressive methylprednisolone reduced the secretion of new inflammatory cytokines. Benefiting from its efficient and broad-spectrum capability in reducing the level of cytokines, this core/shell-structured microcomposite suppressed the recruitment of macrophages and protected the injured spinal cord, leading to an improved recovery of motor function. Overall, the designed microcomposite successfully achieved the two-pronged strategy in cytokine neutralization, providing an alternative approach to inhibit neuroinflammation in the injured spinal cord.Peer reviewe

Solid Electrolytic Substrates for High Performance Transistors and Circuits

Ionic liquids/gels have been used to realize field-effect-transistors (FETs) with two dimensional (2D) transition metal dichalcogenides (TMDs) [1]. Although near ideal gating has been reported with this biasing scheme, it suffers from several issues such as, liquid nature of the electrolyte, its humidity dependency and freezing at low temperatures [2]. Recently, air-stable solid electrolytes have been developed, thanks to the advancement in battery technology [3]. Although insulator-to-metal transition has been reported, the realization of 2D TMD FETs on solid electrolytic substrate has not been reported so far to the best of our knowledge [4]. In this work, we demonstrate a lithium ion (Liion) solid electrolytic substrate based TMD transistor and a CMOS amplifier, with near ideal gating efficiency reaching 60 mV/dec subthreshold swing, and amplifier gain ~34, the highest among comparable inverte

Lithium-ion electrolytic substrates for sub-1V high-performance transition metal dichalcogenide transistors and amplifiers

Electrostatic gating of two-dimensional (2D) materials with ionic liquids (ILs), leading to the accumulation of high surface charge carrier densities, has been often exploited in 2D devices. However, the intrinsic liquid nature of ILs, their sensitivity to humidity, and the stress induced in frozen liquids inhibit ILs from constituting an ideal platform for electrostatic gating. Here we report a lithium-ion solid electrolyte substrate, demonstrating its application in high-performance back-gated n-type MoS2 and p-type WSe2 transistors with sub-threshold values approaching the ideal limit of 60 mV/dec and complementary inverter amplifier gain of 34, the highest among comparable amplifiers. Remarkably, these outstanding values were obtained under 1 V power supply. Microscopic studies of the transistor channel using microwave impedance microscopy reveal a homogeneous channel formation, indicative of a smooth interface between the TMD and underlying electrolytic substrate. These results establish lithium-ion substrates as a promising alternative to ILs for advanced thin-film devices

A new species of forest hedgehog (Mesechinus, Erinaceidae, Eulipotyphla, Mammalia) from eastern China

The hedgehog genus Mesechinus (Erinaceidae, Eulipotyphla) is currently comprised of four species, M. dauuricus, M. hughi, M. miodon, and M. wangi. Except for M. wangi, which is found in southwestern China, the other three species are mainly distributed in northern China and adjacent Mongolia and Russia. From 2018 to 2023, we collected seven Mesechinus specimens from Anhui and Zhejiang provinces, eastern China. Here, we evaluate the taxonomic and phylogenetic status of these specimens by integrating molecular, morphometric, and karyotypic approaches. Our results indicate that the Anhui and Zhejiang specimens are distinct from the four previously recognized species and are a new species. We formally described it here as Mesechinus orientalis sp. nov. It is the only Mesechinus species occurring in eastern China and is geographically distant from all known congeners. Morphologically, the new species is most similar to M. hughi, but it is distinguishable from that species by the combination of its smaller size, shorter spines, and several cranial characteristics. Mesechinus orientalis sp. nov. is a sister to the lineage composed of M. hughi and M. wangi from which it diverged approximately 1.10 Ma